A variety of conventional technologies exist for recognizing facial expressions. In the facial expression recognition described in the specification of Japanese Patent No. 02840816 (hereinafter referred to as Patent Document 1), neutral reference images are prepared in advance for use in the facial expression recognition. The input images and the neutral reference images prepared in advance are then wavelet-transformed to compute the mean power of frequency signals for each band. Facial expressions are then determined by computing the difference between the respective mean powers.
In addition, in the facial expression recognition described in Japanese Patent Laid-Open No. 2005-56388 (hereinafter referred to as Patent Document 2), after extracting prescribed feature points in advance from neutral reference images used in the facial expression recognition, distances between the feature points, etc., are computed. Facial expressions are then determined by similarly computing the distances between the feature points in the input images and calculating the values of the difference between the respective distances.
Furthermore, Japanese Patent Laid-Open No. 2005-266984 (hereinafter referred to as Patent Document 3) describes a facial expression recognition technology that does not use neutral reference images. In the facial expression recognition described in Patent Document 3, after detecting the corners of the mouth and the corners of the right and left eyes in an image, facial expressions are determined based on the ratio of the long side to the narrow side in a rectangle produced by these four points.
Furthermore, in Japanese Patent No. 2573126 (Patent Document 4), a method is described wherein eyebrow motion and other facial expression elements and facial expression element information are obtained from feature locations and facial expression element codes are computed based on the facial expression element information, whereupon the facial expression element codes are employed in a prescribed conversion formula to compute an emotional value.
However, since the facial expression recognitions described in the above-mentioned Patent Document 1 and Patent Document 2 capture changes relative to a neutral state, they have the following significant advantages in comparison with the facial expression recognition technology described in Patent Document 3, which does not use neutral reference images: (1) they can accommodate individual differences in the location of facial organs such as the eyes, the mouth, etc.; (2) they can capture slight variation in facial expression with a relatively high degree of detection accuracy. On the other hand, however, these processes have the following problems: (1) time and labor is required in order to register the neutral images in advance; (2) facial expressions can be recognized only for registered persons; (3) more memory areas are required because the number of images etc. subject to registration depends on the number of persons subject to facial expression recognition; (4) when the imaging environment during registration and imaging environment during facial expression recognition are different, facial expressions are sometimes impossible to recognize with accuracy due to the difference between the imaging environments; (5) the accuracy of facial expression recognition is greatly affected by the variation in the definition of the term “neutral image” among the users registering the neutral images. The above-described five problems result from the fact that users manually register neutral images in advance.
Moreover, in Patent Documents 3 and 4, individual differences in terms of feature point location on faces, etc. cannot be accommodated and it is difficult to improve accuracy because the facial expressions of individual people are determined using a common standard.